A Risk-Averse Framework for Non-Stationary Stochastic Multi-Armed Bandits

In a typical stochastic multi-armed bandit problem, the objective is often to maximize the expected sum of rewards over some time horizon $T$. While the choice of a strategy that accomplishes that is optimal with no additional information, it is no longer the case when provided additional environment-specific knowledge. In particular, in areas of high volatility like healthcare or finance, a naive reward maximization approach often does not accurately capture the complexity of the learning problem and results in unreliable solutions. To tackle problems of this nature, we propose a framework of adaptive risk-aware strategies that operate in non-stationary environments. Our framework incorporates various risk measures prevalent in the literature to map multiple families of multi-armed bandit algorithms into a risk-sensitive setting. In addition, we equip the resulting algorithms with the Restarted Bayesian Online Change-Point Detection (R-BOCPD) algorithm and impose a (tunable) forced exploration strategy to detect local (per-arm) switches. We provide finite-time theoretical guarantees and an asymptotic regret bound of order $\tilde O(\sqrt{K_T T})$ up to time horizon $T$ with $K_T$ the total number of change-points. In practice, our framework compares favorably to the state-of-the-art in both synthetic and real-world environments and manages to perform efficiently with respect to both risk-sensitivity and non-stationarity

Successful Management of Infertile Patient with Trans-Fundal Uterine Membrane

Background: This case report describes an infertile patient with a rare endometrial cavity pathology diagnosed on hysteroscopy. Case: The patient was a 39-year-old female with primary infertility of 9 years' duration. A diagnosis of a possible T-shaped uterus on a previous hysterosalpingogram was not confirmed on diagnostic hysteroscopy 5 years earlier at a different infertility center, where she had undergone a cycle of in-vitro fertilization with embryo transfer (IVF-ET) but was unable to conceive. At the time of diagnostic hysteroscopy at the current, unit the patient was found to have a T-shaped cavity and a trans-fundal uterine membrane obscuring an arcuate fundus. Hysteroscopic division of this thin membrane was performed successfully, followed by hysteroscopic division of the uterine septum and hysteroscopic metroplasty of her T-shaped uterus. Results: Subsequently, the patient conceived with IVF-ET but had an early miscarriage. A second IVF-ET cycle resulted in resulted in delivery of a healthy male infant at term. Conclusions: This report described a case of an infertile patient with a trans-fundal membrane in association with a uterine anomaly. The discovery of such a membrane and the uterine anomaly described above, and their hysteroscopic surgical correction, may have contributed to the successful reproductive outcome for this patient. (J GYNECOL SURG 29:88)Peer Reviewedhttps://deepblue.lib.umich.edu/bitstream/2027.42/140092/1/gyn.2012.0028.pd

Deep Reinforcement Learning Algorithms for Hybrid V2X Communication: A Benchmarking Study

In today's era, autonomous vehicles demand a safety level on par with aircraft. Taking a cue from the aerospace industry, which relies on redundancy to achieve high reliability, the automotive sector can also leverage this concept by building redundancy in V2X (Vehicle-to-Everything) technologies. Given the current lack of reliable V2X technologies, this idea is particularly promising. By deploying multiple RATs (Radio Access Technologies) in parallel, the ongoing debate over the standard technology for future vehicles can be put to rest. However, coordinating multiple communication technologies is a complex task due to dynamic, time-varying channels and varying traffic conditions. This paper addresses the vertical handover problem in V2X using Deep Reinforcement Learning (DRL) algorithms. The goal is to assist vehicles in selecting the most appropriate V2X technology (DSRC/V-VLC) in a serpentine environment. The results show that the benchmarked algorithms outperform the current state-of-the-art approaches in terms of redundancy and usage rate of V-VLC headlights. This result is a significant reduction in communication costs while maintaining a high level of reliability. These results provide strong evidence for integrating advanced DRL decision mechanisms into the architecture as a promising approach to solving the vertical handover problem in V2X

Regularization of the policy updates for stabilizing Mean Field Games

This work studies non-cooperative Multi-Agent Reinforcement Learning (MARL) where multiple agents interact in the same environment and whose goal is to maximize the individual returns. Challenges arise when scaling up the number of agents due to the resultant non-stationarity that the many agents introduce. In order to address this issue, Mean Field Games (MFG) rely on the symmetry and homogeneity assumptions to approximate games with very large populations. Recently, deep Reinforcement Learning has been used to scale MFG to games with larger number of states. Current methods rely on smoothing techniques such as averaging the q-values or the updates on the mean-field distribution. This work presents a different approach to stabilize the learning based on proximal updates on the mean-field policy. We name our algorithm Mean Field Proximal Policy Optimization (MF-PPO), and we empirically show the effectiveness of our method in the OpenSpiel framework

Gestational Sac Aspiration of Heterotopic Ectopic Pregnancy in a Cesarean Section Scar

Background: This article describes a case of heterotopic pregnancy that included a normal twin intrauterine pregnancy and one cesarean section (CS) scar pregnancy diagnosed at 6 weeks of gestation. Ultrasound-guided aspiration of the ectopic gestational sac was performed, and the concurrent twin intrauterine pregnancy (IUP) was preserved successfully. The patient was a 50-year-old woman with secondary infertility. Case: The patient underwent in vitro fertilization and embryo transfer using a donor-egg program to achieve pregnancy with her current partner. At 6-weeks' gestation, she underwent a transvaginal ultrasound scan (US) examination showing a viable twin IUP with a third gestational sac with viable embryo located low within the anterior wall of the uterus. The appearance was consistent with a cesarean scar ectopic pregnancy. This was confirmed on a subsequent US 1 week later. She desired to continue the intrauterine pregnancy. US-guided aspiration of the cesarean scar ectopic pregnancy was attempted. The treatment was successful. Results: The twin pregnancy progressed without further complications. Conclusions: Heterotopic CS ectopic pregnancy can be successfully treated with transvaginal US-guided aspiration. (J GYNECOL SURG 29:317)Peer Reviewedhttps://deepblue.lib.umich.edu/bitstream/2027.42/140090/1/gyn.2012.0026.pd

Assessment of functional feeding groups (FFG) structure of aquatic insects in North- western Rif - Morocco

The involvement of trait-based approaches is crucial for understanding spatial patterns, energy flow and matter transfer in running water systems, which requires consistent knowledge of the functional structures of aquatic communities, with the advantage of combining physical properties and behavioral mechanisms of food acquisition rather than the taxonomic group. The present study indicated how functional feeding groups may be used as a proxy for classical taxonomic evaluation, as well as the potential interest in incorporating them as indicators of anthropogenic stressors. The composition and abundance of the functional feeding groups of aquatic insects were examined from September 2021 to August 2022 along the Western Rif Region.Benthic samples were collected from nine sampling points in the studied area using a Surber sampler with a mesh size of 500 µm and a diameter of 20*20 cm. The stations included in this work were chosen for their accessibility as well as their position on the hydrographic systems. The abundance of sampled aquatic organisms in the whole study area revealed 5,342 individuals belonging to 60 families and seven orders of aquatic insects, classified into five feeding functional groups. In terms of abundance, Collector-gatherers (Ephemeroptera and Diptera) were the most abundant trophic group at most of the sites, with a proportion of 38.47%. Predators (Coleoptera, Hemiptera and Odonata) were the second group at all sites, followed by Collector-filters, accounting for 39.53%, 28.14% and 22.37% respectively, while Scarpers and Shredders had the lowest representation across all sites with 4.16%. The high number of registered Collectors could be related to their ability to feed on a diverse range of food items compared to the remaining trophic guilds. According to the Canonical Correspondence Analysis results, physicochemical (i.e. T, pH, BOD5, Cl- and NO3-) and hydromorphological (i.e. current velocity and depth) variables were amongst the key predictors of shaping the functional structure of aquatic biota during this investigation. It is highly recommended to carry out suitable measures to largely attenuate anthropogenic pressures in order to preserve the integrity of freshwater bodies and their biota

SOREO: A System for Safe and Autonomous Drones Fleet Navigation with Reinforcement Learning

This demonstration introduces SOREO, a system that explores the possibility of extending UAVs autonomy through machine learning. It brings a contribution to the following problem: Having a fleet of drones and a geographic area, how to learn the shortest paths between any point with regards to the base points for optimal and safe package delivery? Starting from a set of possible actions, a virtual design of a geographic location of interest, e.g., a city, and a reward value, SOREO is capable of learning not only how to prevent collisions with obstacles, e.g., walls and buildings, but also to find the shortest path between any two points, i.e., the base and the target. SOREO exploits based on the Q-learning algorithm

CRISE DE GFA SUR UNE CONFIGURATION IRIS PLATEAU SECONDAIRE AU POLYKYSTOSE IRIDO-CILIAIRE

Introduction :L’Iris plateau est une anomalie de la configuration de l’iris. La biomicroscopie ultrasonore (UBM) constitue une aide précieuse pour son diagnostic ainsi que pour la détermination de son caractère primaire ou secondaire. Nous rapportons le cas d’une configuration iris plateau secondaire à une polykystose irido-ciliaire.Observation :Patient de 34 ans, sans antécédents familiaux de glaucome, vu en urgence pour une crise de GFA de l’oeil gauche. La gonioscopie dynamique de l’oeil Adelphe a montré un aspect en double bosse caractéristique de la configuration iris plateau. Un examen UBM fut alors réalisé et a confirmé le diagnostic de la configuration iris plateau secondaire à une polykystose irido-ciliaire bilatérale. Après le contrôle de la crise aigue, le malade a bénéficié d’une iridotomie périphérique pour ces deux yeux et son tonus oculaire est bien équilibré sous une monothérapie.Discussions :Les kystes irido-ciliaires primaires sont l’une des étiologies responsables de l’iris plateau secondaire. L’UBM constitue une aide diagnostic précieuse, car elle donne une meilleure analyse du corps ciliaire et des structures en arrière de l’iris.Conclusion :La polykystose irido-ciliaire peut être responsable d’une hypertonie oculaire aigue par fermeture de l’angle. L’UBM constitue un examen capital pour évoquer ce diagnostic. Le contrôle de la pression intra-oculaire peut être obtenu par une monothérapie et une IP au laser YAG